Crate opendal

Apache OpenDAL™ is a data access layer that allows users to easily and efficiently retrieve data from various storage services in a unified way.

Quick Start

OpenDAL’s API entry points are Operator and BlockingOperator. All public APIs are accessible through the operator. To utilize OpenDAL, you need to:

• Init a service
• Compose layers
• Use operator

Init a service

The first step is to pick a service and init it with a builder. All supported services could be found at services.

Let’s take services::S3 as an example:

use opendal::services;
use opendal::Operator;
use opendal::Result;

fn main() -> Result<()> {
    // Pick a builder and configure it.
    let mut builder = services::S3::default();
    builder.bucket("test");

    // Init an operator
    let op = Operator::new(builder)?.finish();
    Ok(())
}

Compose layers

The next setup is to compose layers. Layers are modules that provide extra features for every operation. All builtin layers could be found at layers.

Let’s use layers::LoggingLayer as an example; this layer adds logging to every operation that OpenDAL performs.

use opendal::layers::LoggingLayer;
use opendal::services;
use opendal::Operator;
use opendal::Result;

#[tokio::main]
async fn main() -> Result<()> {
    // Pick a builder and configure it.
    let mut builder = services::S3::default();
    builder.bucket("test");

    // Init an operator
    let op = Operator::new(builder)?
        // Init with logging layer enabled.
        .layer(LoggingLayer::default())
        .finish();

    Ok(())
}

Use operator

The final step is to use the operator. OpenDAL supports both async Operator and blocking BlockingOperator. Please pick the one that fits your use case.

Every Operator API follows the same pattern, take read as an example:

• read: Execute a read operation.
• read_with: Execute a read operation with additional options, like range and if_match.
• reader: Create a reader for streaming data, enabling flexible access.
• reader_with: Create a reader with advanced options.

use opendal::layers::LoggingLayer;
use opendal::services;
use opendal::Operator;
use opendal::Result;

#[tokio::main]
async fn main() -> Result<()> {
    // Pick a builder and configure it.
    let mut builder = services::S3::default();
    builder.bucket("test");

    // Init an operator
    let op = Operator::new(builder)?
        // Init with logging layer enabled.
        .layer(LoggingLayer::default())
        .finish();

    // Fetch this file's metadata
    let meta = op.stat("hello.txt").await?;
    let length = meta.content_length();

    // Read data from `hello.txt` with range `0..1024`.
    let bs = op.read_with("hello.txt").range(0..1024).await?;

    Ok(())
}

Modules

• docsdocs
  This module holds documentation for OpenDAL.
• layers
  Layer is the mechanism to intercept operations.
• operator_functions
  Functions provides the functions generated by BlockingOperator
• operator_futures
  Futures provides the futures generated by Operator
• raw
  Raw modules provide raw APIs that used by underlying services
• services
  Services will provide builders to build underlying backends.

Structs

• BlockingLister
  BlockingLister is designed to list entries at given path in a blocking manner.
• BlockingOperator
  BlockingOperator is the entry for all public blocking APIs.
• BlockingReader
  BlockingReader is designed to read data from given path in an blocking manner.
• BlockingWriter
  BlockingWriter is designed to write data into given path in an blocking manner.
• Capability
  Capability is used to describe what operations are supported by current Operator.
• Entry
  Entry returned by Lister or BlockingLister to represent a path and it’s relative metadata.
• Error
  Error is the error struct returned by all opendal functions.
• Lister
  Lister is designed to list entries at given path in an asynchronous manner.
• Metadata
  Metadata carries all metadata associated with a path.
• Operator
  Operator is the entry for all public async APIs.
• OperatorBuilder
  OperatorBuilder is a typed builder to build an Operator.
• OperatorInfo
  Metadata for operator, users can use this metadata to get information of operator.
• Reader
  Reader is designed to read data from given path in an asynchronous manner.
• Writer
  Writer is designed to write data into given path in an asynchronous manner.

Enums

• EntryMode
  EntryMode represents the mode.
• ErrorKind
  ErrorKind is all kinds of Error of opendal.
• Metakey
  Metakey describes the metadata keys that can be stored or queried.
• Scheme
  Services that OpenDAL supports

Traits

• Builder
  Builder is used to set up a real underlying service, i.e. storage accessor.

Type Aliases

• Result
  Result that is a wrapper of Result<T, opendal::Error>